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EXECUTIVE  SUMMARY 


OBJECTIVE 

Sunlight  reflected  on  the  moving  sea  surface,  often  referred  to  as  glitter,  is  usually  considered  a  hin¬ 
drance  to  analysis  of  ocean  scenes,  and  glitter  removal  algorithms  arc  often  employed  in  pre-processing 
of  ocean  imagery.  However,  glitter  statistics  can  provide  valuable  insight  into  sea  surface  behavior,  both 
from  the  standpoint  of  characterizing  image  “clutter”  and  gaining  a  better  understanding  of  sea  surface 
roughness.  In  this  work,  sun  glitter  is  observed  through  shore-based  video  of  the  ocean  collected  at  vari¬ 
ous  optical  wavelengths  (visible  and  infrared),  and  various  statistical  properties  of  the  collected  video  arc 
analyzed. 


RESULTS 

A  MATLAB®  tool  has  been  developed  with  which  to  analyze  several  key  statistics  of  the  collected 
video.  This  report  details  the  functionality  of  this  tool  and  demonstrates  the  behavior  of  a  number  of  spatial-, 
temporal-,  and  spectral-domain  statistics  of  glitter  video.  A  detailed  background  on  previous  glitter-related 
research  is  also  given. 


RECOMMENDATIONS 

Additional  research  is  required  to  fully  determine  the  possible  applications  of  this  work.  Data  collec¬ 
tions  in  a  more  controlled  environment  would  enable  better  understanding  of  the  impact  of  various  collec¬ 
tion  variables  on  statistical  behavior. 
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1.  INTRODUCTION 


Glitter  analysis  through  image  processing  goes  back  at  least  to  1951  when  Charles  Cox  and  Walter 
Munk  took  photographs  of  sun  glitter  patterns  over  the  Pacific  Ocean  using  cameras  in  the  bomb  bay  of  a 
World  War  II  surplus  B-17G  aircraft.  They  derived  a  wave-slope  probability  density  function  for  the  glitter 
patterns  by  comparing  the  photographic  density  to  the  probability  of  a  sun  glitter  wave  slope.  In  the  sub¬ 
sequent  decades,  further  analysis  of  the  geometric  and  statistical  properties  of  glitter  has  been  performed 
using  still  images  of  sun  glitter  as  well  as  glitter  synthesized  with  light  sources  such  as  lasers.  Such  analy¬ 
sis  can  reveal  information  about  ocean  roughness  and  wind  speed.  Glitter  is  often  considered  a  hindrance 
to  analysis  of  ocean  scenes  and  glitter  removal  algorithms  are  often  employed  in  pre-processing  of  ocean 
imagery.  However,  as  demonstrated  over  years  of  research,  glitter  statistics  can  also  provide  insight  into 
the  properties  and  motion  of  the  sea  surface. 

This  research  project  is  an  empirical  study  of  the  spatial  as  well  as  temporal  characteristics  of  sun 
glitter  reflected  on  the  ocean  surface,  and  the  spatial  variability  of  statistical  measures  (such  as  tempo¬ 
ral  power  spectral  density)  driven  by  spatial  variations  in  environmental  conditions.  Glitter  is  observed 
through  shore -based  video  of  the  ocean  collected  at  various  wavelengths  (visible  and  infrared).  The  sta¬ 
tistical  properties  of  glitter  can  vary  at  each  pixel  in  a  video  according  to  many  factors,  including  surface 
wave  conditions,  wind  speed,  biological  surfactants,  and  influence  of  nearby  vessels.  This  research  has 
numerous  potential  applications  for  intelligence,  surveillance,  and  reconnaissance  (ISR),  such  as  for  port 
monitoring  and  other  surveillance  systems.  Research  into  glitter  statistics  will  be  succeeded  by  algorithm 
development  in  support  of  these  various  applications. 

The  research  phase  of  this  project  consists  of  statistical  analysis  of  an  initial  data  collection  of  ocean 
video.  Data  collected  under  a  variety  of  conditions  will  provide  greater  understanding  of  the  effect  of  the 
surveillance  system  on  glitter  observations.  Additional  data  will  be  collected  to  incorporate  ocean  behavior 
models  into  the  statistical  analysis.  As  the  behavior  of  the  collected  data  is  better  understood,  so  is  our 
ability  to  develop  automated  detection  tools  for  improved  ISR  and  overall  maritime  domain  awareness. 
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2.  PREVIOUS  WORK 


2.1  EARLY  STUDIES  OF  GLITTER 

The  behavior  of  light  sparkling  on  water,  often  referred  to  as  glitter,  has  long  been  a  subject  of  interest 
to  artists  and  scientists  alike.  Many  authors  have  examined  glitter  and  considered  relationships  between 
glitter  geometry  and  the  surface  slopes  of  the  body  of  water.  In  1822,  a  letter  was  written  by  Spooner  [1] 
reporting  measurements  of  the  width  of  the  glitter  pattern  in  the  Tyrrhenian  Sea,  indicating  a  maximum 
surface  slope  of  25  degrees.  Measurements  of  glitter  patterns  over  the  Black  Sea  were  recorded  by  Shuleikin 
[2],  Many  early  qualitative  descriptions  of  light  reflected  on  water  exist  as  well  [3,4], 

Photographs  have  for  many  years  been  used  to  remotely  sense  ocean  behavior.  Experiments  performed 
by  researchers  at  the  Naval  Research  Laboratory  (NRL)  and  documented  by  Hulburt  [5]  studied  the  effects 
of  polarized  light  on  the  ability  to  view  the  horizon  or  objects  on  the  ocean  surface.  They  used  polarizing 
attachments  to  a  sextant  and  binoculars  to  improve  the  performance  of  those  instruments,  and  recorded 
the  width  of  glitter  pattern  (as  determined  by  photographs)  versus  sun  elevation  and  wind  velocity.  The 
seminal  photographic  glitter  study  was  conducted  by  Cox  and  Munk  [6],  with  funding  from  the  Air  Force 
Research  Center  [see  also  7-9].  The  authors  collected  aerial  photographs  over  the  open  ocean  and  deduced 
that  the  distribution  of  wave  slopes  is  approximately  Gaussian. 

Motivated  by  the  work  of  Cox  and  Munk  [6]  and  Hulburt  [5],  NRL  scientist  Schooley  [10]  collected 
photographs  of  glitter  patterns  resulting  from  a  flash  bulb  rather  than  the  sun  (“flash-sparkle  pictures”). 
Quantitative  information  was  extracted  from  the  pictures  by  computing  the  area  of  glitter  which  appeal's 
in  square  regions  of  the  photographs  and  using  the  areas  to  construct  a  wave-slope  probability,  which  is 
shown  (as  in  [6])  to  be  approximately  Gaussian.  The  standard  deviation  of  the  wind  slope  is  plotted  against 
wind  velocity.  Data  from  [5]  was  shown  to  agree  with  the  flash-sparkle  data.  Work  by  Barber  [1 1]  used 
a  photographic  set-up  to  measure  wave  direction  by  the  correlogram.  Later  work  from  NRL  [12,  13]  de¬ 
veloped  a  technique  for  determining  the  directional  energy  spectrum  by  means  of  an  optical  Fourier  trans¬ 
form. 


2.2  LEGACY  OF  COX  AND  MUNK 

Largely  inspired  by  the  work  of  Cox  and  Munk,  many  theoretical  studies  have  been  written  on  the  be¬ 
havior  of  glitter  and  how  it  can  be  exploited  in  imagery.  A  series  of  papers  by  Longuet-Higgins  [14,  15, 

16]  provides  a  geometric  analysis  of  the  creation  and  annihilation  of  points  of  light  (“twinkles”)  on  a  mov¬ 
ing  surface.  This  work  continues  to  be  relevant;  studies  of  the  statistics  of  specular  point  phenomenology, 
parallelling  the  work  of  Longuet-Higgins,  were  published  decades  later  by  Akhmedov,  Gardashov,  and 
Shifrin  [17]  and  Gardashov  [18]. 

A  number  of  authors  have  continued  the  analysis  of  Stilwell  [12]  to  determine  the  wave  spectrum  from 
a  photograph.  First-  and  higher-order  analyses  of  spectrum  computation  from  an  optical  Fourier  transform 
under  various  sky  conditions  were  given  by  Kasevich,  Tang,  and  Henriksen  [19]  and  Kasevich  [20].  An 
alternate  approach  to  that  taken  by  Stilwell  [12]  is  proposed  by  Peppers  and  Ostrem  [21].  This  new  ap¬ 
proach  is  not  limited  by  small  wave  slope  approximation,  and  is  developed  for  several  sky  radiance  mod¬ 
els.  A  report  by  Bjerkaas  and  Riedel  [22]  out  of  The  Johns  Hopkins  University  Applied  Physics  Labora¬ 
tory  (JHU/APL)  develops  a  new  model  for  the  elevation  spectrum  of  wind-generated  ocean  waves,  mod¬ 
ifying,  simplifying,  and  correcting  errors  from  the  models  developed  at  NASA  by  Pierson  and  Stacy  [23] 
and  Pierson  [24].  The  work  of  Stilwell  [12]  is  extended  by  several  authors  from  JHU/APL  in  [25,  26]  and 
Chapman  and  Irani  [27]. 
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A  procedure  for  estimating  unpolarized  irradiance  reflectance  and  glitter  patterns  as  a  function  of  light¬ 
ing  and  wind  speed  was  given  by  Preisendorfer  and  Mobley  [28,  29].  Tse,  McGill,  and  Kelly  [30]  gener¬ 
ated  simulated  whitecap  and  glitter  radiance  images,  motivated  by  a  need  to  remove  effects  of  whitecaps  or 
glitter,  or  to  use  as  ground  truth  to  verify  procedures  for  deducing  wave  slopes,  using  the  wave  spectrum 
developed  by  Bjerkaas  and  Riedel  [22].  Zeisse  [31,  32] — Reports  out  of  the  Naval  Command,  Control 
and  Ocean  Surveillance  Center  (NCCOSC — now  SSC  Pacific).  [31]  derives  an  integral  equation  predict¬ 
ing  slope  distribution  on  the  horizon,  extending  the  work  of  [6].  [32]  documents  FORTRAN  code  based 
on  the  Cox-Munk  model  to  predict  the  radiance  of  the  ocean  surface.  A  later  SSC  San  Diego  (now  SSC 
Pacific)  report  also  by  Zeisse  [33]  studies  grazing  optical  reflectivity  over  capillary  waves.  In  work  pro¬ 
duced  through  the  Naval  Command,  Control  and  Ocean  Surveillance  Center  (NCCOSC)  and  SSC  San 
Diego  (both  now  SSC  Pacific),  Zeisse  [31,  32,  33]  derived  an  integral  equation  predicting  slope  distribu¬ 
tion  on  the  horizon,  extending  the  work  of  Cox  and  Munk  [6] ;  documented  FORTRAN  code  based  on  the 
Cox-Munk  model  to  predict  the  radiance  of  the  ocean  surface;  and  studied  grazing  optical  reflectivity  over 
capillary  waves.  Elfouhaily,  Chapron,  Katsaros,  and  Vandemark  [34]  developed  a  new  analytical  spec¬ 
trum  model  featuring  wave  age  dependency  in  both  long-  and  short-wave  formulations,  using  data  from  the 
Joint  North  Sea  Wave  Project  (JONSWAP)  [35]  to  formulate  the  model.  That  work  also  summarizes  and 
addresses  shortcomings  of  some  earlier  spectrum  models,  including  that  proposed  in  [22]. 

Two  English-language  research  groups  have  continued  publishing  glitter  studies  into  the  21st  century. 
Alvarez-Borrego  and  collaborators  have  published  a  number  of  papers  in  the  past  three  decades  [  36 — 45  ] . 
Relationships  have  been  derived  between  the  autocorrelations  of  surface  wave  hights  and  glitter  patterns; 
these  relationships  are  then  inverted  so  that  wave  heights  can  theoretically  be  obtained  from  aerial  pho¬ 
tographs  of  glitter  patterns.  Wave  height  spectra  arc  then  calculated  via  Fourier  transform.  The  theory  is 
compared  to  experimental  data.  The  various  papers  cited  present  derivations  in  one  and  two  dimensions 
and  under  other  various  conditions,  and  conditions  under  which  inversion  is  possible  are  discussed.  A  se¬ 
ries  of  papers  by  Cureton  [46-49]  corrects  the  model  from  [39],  applies  the  model  to  real  data,  and  extends 
the  theory  to  a  higher  order.  Additionally,  Weber  [50]  gives  theoretical  discussion  of  an  imaging  system  for 
viewing  objects  underwater  through  rough  seas  (most  citations  arc  in  Russian). 


2.3  PHOTOGRAPHIC  DATA  COLLECTIONS 

Many  photographic  experiments  and  measurement  collections  have  been  made  since  the  work  of  Cox 
and  Munk  and  have  been  used  to  further  glitter  analysis.  Hughes  and  Grant  [51]  took  a  collection  of  pho¬ 
tographs,  wave  slopes,  and  other  measurements  taken  of  a  ship  wake  in  “dead  water.”  Theoretical  analyses 
of  this  scenario  arc  given  by  Hughes  [52].  The  spectrum  derivation  with  simplified  calculations  is  detailed 
by  Gotwols  and  Irani  [53].  Lubard,  Krimmel,  and  Thebaud  [54]  determined  wave  number  spectra  and 
space-time  spectra  from  optical  video  data  collected  during  the  West  Coast  Experiment  [see  55]  conducted 
by  the  Jet  Propulsion  Laboratory  (JPL)  at  the  Naval  Ocean  Systems  Center  (NOSC — now  SSC  Pacific). 
Saunders  [56],  Gambling  [57],  and  Fraedrich  [58]  conducted  radiometer  measurements  of  sea  glitter  in  the 
infrared  range,  comparing  field  measurements  with  model  calculations  such  as  those  in  [6].  The  work  by 
Fraedrich  [58]  emphasizes  low  observation  angles  and  contains  information  about  the  temporal  properties 
of  glitter  derived  from  power  spectra.  Gasparovic  and  Etkin  [59]  documents  the  Joint  U.S./Russia  Inter¬ 
nal  Wave  Remote  Sensing  Experiment  that  collected  remote  sensing  data,  including  optical  cameras,  along 
with  meteorological  and  oceanographic  (METOC)  instruments  for  in  situ  data.  More  recently,  Strizhkin 
[60]  performed  an  optical  analysis  in  the  style  of  Cox  and  Munk  [6]  on  low-altitude  photographs  of  the 
sea. 
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Technological  advances  since  the  1950s  have  enabled  data  to  be  collected  in  the  time  domain  as  well. 
Video  images  of  the  ocean  surface  were  taken  by  [61]  from  an  offshore  platform  and  used  to  compute  an 
ensemble-averaged  space-time  spectrum,  extending  the  techniques  from  [11-13,  20]  to  the  time  domain. 
Shaw  and  Churnside  [62]  calculated  slope  probability  density  functions  from  glint  counts  from  a  scanning 
laser  over  the  Pacific  Ocean,  and  in  [63]  described  fractal  behavior  exhibited  by  the  time  series  of  the  laser 
glint  counts.  Titov  et  al.  [64]  and  Titov,  Zuikova,  Luchinin,  and  Troitzkaya  [65]  studied  spatial  and  tempo¬ 
ral  spectral  behavior  of  surface  waves  over  the  Black  Sea  using  an  optical  spectrum  analyzer. 

A  number  of  controlled  experiments  have  been  performed  in  wind-wave  tanks  under  various  set-ups, 
including  those  described  in  [66-77], 

Other  forms  of  still  imagery  have  been  used  to  study  glitter,  including  stereo  photos  [78-86]  (as  dis¬ 
cussed  by  Jahne,  Klinke,  and  Waas  [87]  and  Monaldo  and  Kasevich  [25]),  polarimetric  imagery  [88],  in¬ 
frared  imagery  [89-91],  satellite  imagery  [92-95],  and  other  imagery  types  [96]. 


2.4  INTRODUCTORY  REFERENCES  TO  GLITTER 

Several  works  contain  introductory  sections  with  useful  background  information  about  earlier  glitter 
research.  Cox  [97,  chapter  3],  discusses  in  depth  the  work  of  Cox  and  Munk  [6]  and  Longuet-Higgins  [14, 
15,  16],  Gregg  [98]  surveys  experiments  and  results  in  turbulent  mixing  since  the  1950s.  Jahne,  Klinke, 
and  Waas  [87]  provides  a  summary  of  optical  techniques  for  wave  slope  imaging,  including  stereophotog¬ 
raphy,  as  well  as  a  theoretical  review  of  computer  vision  techniques  employed  for  this  problem.  Walker 
[99,  chapter  7]  provides  sun  glitter  theory,  emphasizing  the  theory  of  specular  points  by  Longuet-Higgins 
[14,  15,  16],  and  glitter  autocorrelation  and  spectral  density  motivated  by  Titov  [100],  An  appendix  to  the 
chapter  details  the  results  of  Cox  and  Munk  [6],  Lynch,  Dearborn,  and  Lock  [101]  give  a  qualitative  de¬ 
scription  of  glitter,  including  a  summary  of  earlier  work  and  experiments  with  short-  and  long-time  expo¬ 
sure  photographs  and  high  frame  rate  videos. 
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3.  DATA  COLLECTION  AND  PROCESSING 


3.1  VIDEO  CAPTURE 

Glitter  behavior  has  been  observed  visually  and  captured  on  video  under  various  conditions.  Collec¬ 
tions  took  place  on  Point  Loma  in  San  Diego,  California,  at  various  elevations,  times  of  day,  sea  states, 
and  meteorological  conditions.  To  a  human  observer,  glitter  can  look  wildly  different  under  different  con¬ 
ditions.  Sea  state  affects  glitter  patterns  by  changing  the  course  of  wave  slopes,  and  glitter  on  a  relatively 
flat  sea  surface  has  a  different  character  than  glitter  appealing  on  the  face  of  a  wave.  When  the  sun  is  high 
overhead,  glitter  appeal's  glittery,  sparkly,  and  rapidly  changing,  whereas  a  sun  low  on  the  horizon  causes 
more  specular  reflection  and  a  more  static  glitter  pattern.  Even  within  a  single  scene,  the  appearance  of 
glitter  can  vary.  Certain  patches  of  ocean  reflect  sunlight  differently,  possibly  due  to  the  presence  of  sea 
weed  or  other  debris,  and  reflections  tend  to  be  denser  closer  to  the  horizon  and  extremely  disparate  in  the 
waves  near  the  shore.  Glitter  behavior  is  different  near  the  edges  of  the  larger  glitter  pattern  than  near  the 
center,  and  cloud  cover  affects  the  size  and  shape  of  glitter  patterns  on  the  water. 

Glitter  captured  on  camera  can  differ  in  other  ways.  For  this  analysis,  we  have  collected  video  in  vis¬ 
ible  and  near  infrared  (NIR)  wavelengths  using  various  zoom  lenses  and  at  two  different  look  angles  (ele¬ 
vations).  Given  the  large  number  of  variables  involved,  this  collection  is  intended  to  provide  a  sampling  of 
glitter  behavior,  rather  than  a  tightly  controlled  experiment.  Analysis  of  the  several  videos  collected  will 
inform  the  direction  and  design  of  future  collections  and  experiments. 

•  Low  elevation,  zoomed  in  (visible  only) — three  regions  of  interest  (ROIs),  three  ROI  sizes,  unpro¬ 
cessed  and  processed 

•  Low  elevation — visible  and  NIR,  unprocessed  and  processed 

•  High  elevation — visible  and  NIR,  unprocessed  and  processed 


3.2  GRAPHICAL  USER  INTERFACE 

A  graphical  user  interface  (GUI)  analysis  tool  was  developed  in  MATLAB,  which  enables  users  to 
read,  process,  and  analyze  glitter  videos.  See  Figure  1 .  The  tool  can  be  easily  modified  to  accommodate 
additional  analysis  requirements.  This  section  serves  as  a  user  guide  for  the  tool,  detailing  its  various  fea¬ 
tures.  Contact  the  author  for  access  to  the  software. 

3.2.1  Data  Processing 

3.2. 1. 1  Data  Import.  The  Glitter  GUI  provides  two  options  for  importing  data. 

•  Select  Video  enables  the  user  to  select  a  local  video.  Upon  selection,  the  video’s  path  and  filename 
as  well  as  its  first  frame  are  displayed  on  the  GUI. 

•  Select  Data  enables  the  user  to  select  a  .mat  file  that  has  been  exported  and  saved  previously  by  the 
GUI.  This  is  useful  for  generating  additional  analysis  on  a  data  set  without  needing  to  repeatedly 
process  the  video  (which  can  be  time  consuming). 
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Figure  1 .  A  screen  shot  of  the  MATLAB  GUI  for  glitter  analysis. 

3.2. 1.2  Data  Export.  After  a  video  is  selected,  pre-processing  steps  can  be  applied  to  a  selected  spa- 
tiotemporal  block. 

The  Process  Frame  box  contains  several  options  for  pre-processing  the  selected  video. 

•  Channel  indicates  which  video  channel  to  process  (usually  1,  2,  or  3 — default  is  1). 

•  Downsample  Factor  selects  a  factor  between  0  and  1  by  which  to  downsample  (default  is  1 — no 
downsampling). 

•  Filter  Type  applies  averaging  or  a  Gaussian  filter  (default  is  no  filter). 
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•  hsize  specifies  the  size  of  the  filter.  This  can  be  a  vector  specifying  the  number  of  rows  and  columns 
in  the  filter,  or  a  scalar,  in  which  case  the  filter  is  a  square  matrix  (default  is  [3  3]).  This  option  has 
no  effect  if  no  filter  is  selected. 

•  sigma  is  a  positive  scalar  specifying  standard  deviation  of  the  Gaussian  filter.  This  option  has  no 
effect  if  average  or  no  filter  is  selected. 

The  Preview  button  applies  the  selected  parameters  to  the  first  frame  of  the  video  and  displays  the 
processed  frame.  The  Reset  button  resets  all  parameters  to  their  defaults  and  removes  any  spatial  cropping 
that  has  been  applied. 

A  spatiotemporal  ROI  in  the  video  can  be  selected  for  analysis.  Several  options  arc  available  for  crop¬ 
ping  a  spatial  region. 

•  Click  Crop  Frame  (with  the  Fixed  ROI  box  unchecked)  and  a  new  frame  will  open  displaying 
frame  1  of  the  video.  Select  two  opposite  corners  of  the  desired  rectangular  region. 

•  Select  Fixed  ROI  and  enter  the  desired  dimensions  in  the  height  and  width  boxes.  If  the  center 
boxes  arc  left  empty,  or  the  center  point  is  not  within  the  dimensions  of  the  video,  a  window  will 
open  allowing  the  user  to  graphically  select  the  center  of  the  ROI. 

Once  a  spatial  ROI  is  selected,  the  first  frame  of  this  region  is  displayed  in  the  GUI.  Additional  pre¬ 
processing  may  be  applied  to  this  region.  The  Select  Frames  boxes  allow  the  user  to  specify  which  frames 
of  the  video  will  be  processed. 

Once  pre-processing  steps  and  a  ROI  are  selected,  they  must  be  applied  to  every  frame  of  the  video. 
This  can  take  several  minutes  or  longer,  depending  on  the  size  of  the  selected  block.  The  frame-by-frame 
processing  will  occur  only  once,  upon  selection  of  Export  and  Save,  or  any  of  the  graphing  options.  It  is 
recommended  that  Export  and  Save  be  selected  before  any  graphs  arc  displayed,  especially  if  the  selected 
block  will  be  analyzed  further  in  the  future.  When  Export  and  Save  is  pressed,  the  frames  are  processed 
and  once  completed  the  user  will  be  prompted  to  save  a  .mat  file  containing  the  processed  data.  This  .mat 
file  can  be  loaded  in  a  future  session  with  the  Select  Data  option.  The  data  contained  in  the  saved  .mat  file 
are: 

•  info  is  a  structure  documenting  video  metadata  and  preprocessing  parameters; 

•  FRAMES  is  a  cell  (size  1  x  number-of-frames)  containing  each  processed  frame; 

•  TS  is  a  cell  (with  the  same  dimensions  as  ROI)  containing  the  time  series  obtained  at  each  pixel;  and 

•  PS  is  a  cell  (with  the  same  dimensions  as  ROI)  containing  the  power  spectral  density  (PSD)  of  the 
time  series  at  each  pixel. 

3.2.2  Data  Visualization 

Many  options  are  available  for  visualizing  the  selected  ROI.  The  left-hand  box  displays  graphs  of  data 
taken  from  the  entire  spatiotemporal  block.  When  Display  Graphs  is  pressed,  the  selected  graphs  will 
open  in  individual  figures.  Many  of  these  graphs  will  be  difficult  to  read  for  large  ROIs.  The  right-hand 
box  displays  single  frames  of  the  ROI  or  time  series  or  PSDs  at  individually  selected  pixels. 

•  Time  Graph  displays  the  time  series  at  every  pixel  on  one  set  of  axes.  Options  exist  to  overlay  me¬ 
dian  line  or  to  detrend  each  time  series. 
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•  Time  Series  Trend  Lines  displays  trend  lines  extracted  from  each  time  series  using  MATLAB 
detrend  function. 

•  Temporal  Means/Temporal  Variances  displays  color  maps  of  the  means  and  variances  of  the  time 
series  at  each  pixel.  If  both  boxes  arc  selected,  the  two  color  maps  show  on  a  single  figure. 

•  Spatial  Means/Spatial  Variances  plots  the  spatial  means  and  variances  of  each  frame.  If  both  boxes 
are  selected,  the  two  plots  show  on  a  single  set  of  axes  (with  two  y-axes). 

•  Temporal  Covariance  displays  a  color  map  of  the  covariance  of  the  frames  over  time. 

•  Power  Spectra  displays  the  PSD  functions  at  each  pixel  on  one  set  of  axes.  Options  exist  to  remove 
means  or  trends  from  each  graph,  or  to  put  either  axis  on  a  log  scale. 

•  Glitter  Count  Time  Series/Glitter  Count  Histogram  counts  the  number  of  glitter  pixels  in  each 
frame  and  displays  a  time  series  or  histogram  of  the  counts.  If  both  arc  selected,  they  appeal-  in  the 
same  figure.  Options  exist  to  set  the  intensity  value  threshold  for  defining  glitter,  and  the  number  of 
bins  for  the  histogram. 

•  View  Single  Time  Series  displays  the  time  series  at  the  selected  pixel,  with  options  to  remove  mean 
or  trend. 

•  View  Single  PSD  displays  the  PSD  function  of  the  time  series  at  the  selected  pixel,  with  options  to 
remove  mean  or  trend  and  to  put  either  axis  on  a  log  scale. 

•  View  Single  Frame  displays  a  specified  frame  of  the  ROI. 

3.2.3  Additional  Functionality 

The  Glitter  GUI  was  made  with  the  GUIDE  function  in  MATLAB,  which  makes  editing  the  GUI  fairly 
straightforward.  Graph  functions  can  be  edited  or  added  to  suit  individual  needs. 


4.  DATA  ANALYSIS 


There  is  significant  variability  in  conditions  under  which  glitter  can  be  observed  and  analyzed.  Envi¬ 
ronmental  variables  (e.g.,  wind  speed,  sun  elevation,  water  temperature),  collection  variables  (e.g.,  cam¬ 
era  and  lens  type,  camera  angle,  and  elevation),  and  processing  variables  (e.g.,  video  channel,  ROI,  image 
smoothing,  and  downsampling)  can  all  affect  observed  glitter  statistics.  While  many  of  these  variables  are 
difficult  to  control  experimentally,  the  effects  of  several,  particularly  collection  and  processing  variables, 
arc  studied  in  this  report. 


4.1  SET-UP  AND  DATA  ACQUISITON 

The  effects  of  several  processing  variables  are  studied  by  analyzing  a  single  video.  This  video,  referred 
to  as  DV 14,  was  collected  on  26  April  2012  at  the  Tidepools  (Monument)  area  of  SSC  Pacific,  at  an  ele¬ 
vation  of  approximately  9  m.  Video  was  recorded  in  the  visible  spectrum  at  30  frames  per  second  with  a 
Canon  7D  and  Opteka  800  mm  f/8  telephoto  mirror  lens  on  a  tripod,  pointed  at  a  heading  of  283°  west. 

Sea  state  and  wind  speeds  were  calm,  and  the  sky  was  partly  cloudy.  Local  time  was  17:37,  at  which  time 
the  sun’s  azimuth  and  elevation  was  approximately  254°  and  47°,  respectively.  The  video  is  2  min  long, 
with  3632  frames.  A  frame  from  DV  14  is  shown  in  Figure  2.  At  full  resolution,  jitter  caused  by  camera 
vibration  is  evident;  to  mitigate  this  effect  and  simplify  analysis,  the  video  is  spatially  downsampled  by  a 
factor  of  2,  and  a  3-by-3  low-pass  Gaussian  filter  with  standard  deviation  of  0.5  pixel  (or  1  pixel  full  width, 
half  max)  is  applied.  All  analysis  on  DV  14  in  this  report  is  done  after  this  processing,  unless  otherwise 
specified. 


Figure  2.  A  frame  from  video  DV14. 
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The  scene  shown  in  DV 14  is  probably  more  complex  than  what  would  be  observed  in  open  ocean,  and 
more  representative  of  many  near-shore  environments.  Due  to  the  large  telephoto  lens,  DV14  provides  a 
clear,  close-up  view  of  glitter  behavior  coming  from  a  mix  of  gravity  and  capillary  surface  waves.  Within 
the  field  of  view  are  three  particular  regions  with  distinct,  persistent  differences  in  glitter  characteristics. 

By  focusing  our  analysis  on  each  of  these  (named)  regions,  we  can  begin  to  study  statistical  variability  of 
glitter  as  actually  encountered  in  nature. 

•  The  Loops  region  is  characterized  by  higher  amplitude  waves  with  glints  appealing  and  disappear¬ 
ing  on  wave  faces,  sometimes  forming  loops  (as  described  by  Longuet-Higgins). 

•  The  Sparkles  region  is  characterized  by  lower  waves  and  many  small,  high  frequency  glints. 

•  The  Flat  region  characterized  by  what  appeal-  to  be  longer  wavelength  waves  and  resulting  large 
reflective  regions. 

The  cause  of  the  persistent  differences  between  these  regions  is  unknown,  but  it  is  likely  a  variation  of  bio¬ 
logical  surfactants  (which  are  expected  to  modulate  capillary  wave  height  and  wavelength)  and  debris  such 
as  seaweed.  Spatially  varying  sky  conditions  may  also  contribute  to  spatial  glitter  variability  in  the  scene. 

ROIs  are  selected  in  each  region.  The  viewing  geometry  in  the  experiment  (owing  to  low  elevation  of 
the  camera),  the  field  of  view  includes  a  wide  range  of  grazing  angles  and  ranges  to  the  sea  surface.  Thus, 
the  particular  size  and  shape  of  the  ROI  should  have  a  significant  impact  on  the  analysis.  Three  different 
shapes  of  ROIs — 16-by-64  pixels,  16-by-16  pixels,  and  l-by-16  pixels — are  selected  centered  at  points 
selected  from  the  three  visual  regions  of  the  video.  The  nine  selected  ROIs  are  shown  in  Figure  3. 


4.2  GLITTER  STATISTICS 

4.2.1  Time  Domain  Statistics 

A  starting  point  for  glitter  video  analysis  is  to  observe  the  time  series  of  image  intensity  at  each  ROI 
pixel.  Time  series  at  pixels  from  each  of  the  three  visual  regions  of  DV14  are  shown  in  Figure  4. 

To  examine  the  data  in  a  three  dimensional  spatiotemporal  block,  the  time  series  associated  to  every 
pixel  in  an  ROI  can  be  graphed  on  the  same  axis,  in  what  is  here  referred  to  as  a  time  graph.  The  time 
graphs  for  the  three  ROIs  in  the  Sparkles  region  are  shown  in  Figures  5  through  7.  Time  graphs  typically 
contain  too  much  data  to  comprehend  visually,  especially  for  larger  ROIs,  but  they  can  convey  the  overall 
behavior  in  a  ROI  or  potentially  indicate  the  presence  of  anomalous  behavior.  The  general  “information 
overload”  in  a  large  ROI  time  graph  illustrates  the  importance  of  looking  to  statistical  analysis  and  fre¬ 
quency  domain  analysis  tools. 

The  information  contained  in  a  time  graph  can  be  summarized  by  graphing  the  change  in  the  spatial 
statistics  of  an  ROI  over  time.  Figure  8  shows  spatial  means  of  the  three  Sparkles  ROIs  graphed  over  time. 
The  graphs  in  Figures  5  through  7,  along  with  Figure  8,  give  a  picture  for  the  general  frame  locations  of 
glitter  within  a  scene. 

4.2.2  Power  Spectral  Density 

Spectral  analysis  of  time  series  gives  an  alternative,  and  often  simpler,  view  of  natural  phenomena. 
The  power  spectral  density  (PSD)  is  a  statistic  of  applied  to  random  processes  that  specifies  the  energy  of 
the  process  over  various  frequencies.  Various  methods  are  used  to  estimate  the  PSD  of  a  signal;  here,  the 
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Figure  3.  Frame  200  of  DV1 4  showing  the  nine  ROIs  selected  in  three  regions. 


Loops  Spukics  Fiat 


Frame  Number  Frame  Number  Frame  Number 

Figure  4.  Example  time  series  at  pixels  selected  from  three  regions  of  DV14:  Loops,  Sparkles,  and 
Flat. 


Welch  method  is  used,  implemented  using  the  MATLAB  function  pwelch.  This  function  calculates  pe- 
riodogram  estimates  of  a  signal  split  into  overlapping  segments  that  are  each  windowed  with  a  Hamming 
window.  The  PSD  estimate  is  obtained  by  averaging  the  modified  periodograms  obtained  from  each  seg¬ 
ment.  Unless  otherwise  specified,  the  PSD  of  a  signal  x  are  calculated  with  pwelch  using  eight  segments 
with  a  50%  overlap,  a  Hamming  window  length  of  5 12,  and  a  sampling  frequency  of  30  Hz.  The  mean 
value  of  x  is  removed  using  the  detrend  function  prior  to  computing  pwelch.  Figure  9  shows  the  PSD 
of  time  series  associated  with  pixels  in  each  of  the  three  regions  in  DV14.  The  corresponding  time  series 
are  shown  in  Figure  4. 
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Figure  5.  Time  graphs  for  a  1 6-by-1 28-pixel  ROI  in  the  Sparkles  region. 
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Figure  6.  Time  graphs  for  a  1 6-by-1 6-pixel  ROI  in  the  Sparkles  region. 


Since  a  PSD  estimate  applies  to  a  single  pixel  in  a  spatiotemporal  region,  all  of  the  PSD  estimates 
from  a  spatial  ROI  can  be  plotted  on  the  same  axes,  analogous  to  the  time  graphs  in  Figures  5  through  7. 
Large  ROIs  contain  too  much  information  to  visualize  well,  so  Figures  10  through  12  show  the  PSD  es- 
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Figure  7.  Time  graphs  for  a  1  -by-1 6-pixel  ROI  in  the  Sparkles  region. 
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Figure  8.  Spatial  means  of  the  three  ROIs  in  the  Sparkles  region  graphed  over  time.  The  dimen 
sions  of  the  ROIs  are,  left  to  right,  1 6-by-1 28,  1 6-by-1 6,  and  1  -by-1 6  pixels. 


timates  from  l-by-16-pixel  ROIs  in  each  of  the  three  regions  of  DV 14.  From  these  graphs,  one  can  see 
which  frequencies  have  the  most  variation  from  pixel  to  pixel  within  a  region. 

4.2.3  Glitter  Counts 

Several  authors  (e.g.,  [44,  45])  have  studied  glitter  in  ocean  imagery  by  considering  a  clipped  image 
revealing  the  locations  of  glitter  pixels — pixels  whose  intensity  is  above  a  given  threshold  value.  Examples 
of  clipped  images  of  a  frame  of  DV14  at  various  threshold  values  are  shown  in  Figure  13. 

Shaw  and  Churnside  [63]  visualize  glitter  behavior  in  video  by  counting  the  number  of  glitter  pixels 
that  appear  in  each  frame.  They  also  compute  histograms  of  these  glitter  counts.  The  experimental  results 
in  that  work  show  statistical  self-similarity  between  glitter  count  time  series  over  varying  time  scales.  The 
authors  draw  a  connection  between  glitter  count  series  and  surface  roughness  variability,  and  go  on  to  fur¬ 
ther  examine  the  fractal  behavior  of  glitter. 
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Power  Spectral  Density  Estir 


Figure  9.  PSD  estimates  from  single  pixels  selected  from  each  of  the  three  regions  of  DV14.  The 
time  series  associated  with  these  same  pixels  are  shown  in  Figure  4. 


Figure  10.  PSD  estimates  from  each  pixel  within  a  1  -by-1 6-pixel  section  of  the  Flat  region  of  DV14. 


The  glitter  studied  in  [63]  was  synthesized  using  scanning  lasers  and  collected  in  open  ocean  from 
an  overhead  view.  In  contrast,  the  data  collected  for  this  work  observes  sun  glitter  in  coastal  waters  at  a 
very  small  viewing  angle.  The  appearance  of  the  glitter  count  series  and  histograms  varies  significantly 
depending  on  the  scene.  Figure  14  shows  the  glitter  count  time  series  and  histogram  for  the  16-by-128- 
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Figure  11.  PSD  estimates  from  each  pixel  within  a  1  -by-1 6-pixel  section  of  the  Loops  region  of 
DV14. 


Figure  12.  PSD  estimates  from  each  pixel  within  a  1 -by-1 6-pixel  section  of  the  Sparkles  region  of 
DV14. 


pixel  Flat  region  in  DV 14.  The  image  is  clipped  using  a  threshold  intensity  value  of  245  and  histogram  is 
constructed  using  500  bins. 
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Figure  13.  Frame  1000  from  DV14  clipped  at  various  threshold  values. 


Glitter  Count  Time  Series  •  Flat  -  16x128  Glitter  Count  Histogram  -  Flat  -  16x128 


(a)  Glitter  count  time  series  (b)  Glitter  count  histogram 

Figure  14.  Glitter  count  time  series  and  histogram  from  a  1 6-by-1 28-pixel  area  of  the  Flat  region  of 
DV14. 


4.2.4  Best-Fit  Analysis 

Each  PSD  in  Figure  9  shows  an  overall  /_7-type  fall-off  trend.  Figures  10  through  12  show  the  great¬ 
est  regional  variation  in  PSD  in  the  2-  to  15-Hz  frequency  range.  By  fitting  a  3  ■  /_7  curve  to  a  PSD,  the 
values  3  and  7  can  be  used  to  characterize  the  PSD  and  compare  it  with  those  from  other  video  pixels.  Fig¬ 
ures  15  through  17  show  the  best-fit  curves  to  PSDs  from  DV 14  under  several  conditions,  all  labeled  with 
the  equations  of  the  best-fit  curves. 
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Power  Spectral  Density  Estir 


Figure  15.  PSDs  and  best-fit  curves  from  pixels  from  each  of  the  three  regions  of  DV14  (the  same 
PSDs  shown  in  Figure  9). 


Figure  16.  PSDs  and  best-fit  curves  from  2  pixels  each  taken  from  the  Sparkles  region  of  DV14, 
one  before  the  video  was  pre-processed  (downsampled  and  Gaussian  blurred)  and  one  after.  Note 
that  unless  otherwise  specified,  all  plots  from  DV14  in  this  report  are  on  pre-processed  data  (see 
Section  4.1). 
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Figure  17.  PSDs  and  best-fit  curves  from  a  Loops  pixel  in  DV14  in  three  separate  color  channels 
(previous  graphs  only  show  data  on  channel  1).  Deviation  is  minimal  between  the  three  PSDs,  and 
the  best-fit  curves  are  nearly  identical. 


18 


5.  CONCLUSION 


As  the  description  in  Section  2  of  previous  research  on  glitter  reveals,  little  is  known  about  the  be¬ 
havior  of  sun  glitter  as  it  appears  in  ocean  video,  and  what  digital  video  processing  techniques  can  reveal 
about  how  ocean  properties  might  be  remotely  sensed  by  such  video.  This  report  has  demonstrated  how 
certain  statistics  can  be  extracted  from  the  glitter  in  ocean  video  and  be  analyzed  and  visualized.  It  has 
been  shown  that  PSD  is  a  strong  function  of  ROI  size,  location,  and  other  parameters.  At  significant  ROI, 
the  PSD  exhibits  a  1/f  type  behavior  which  is  also  spatially  sensitive.  Planned  future  work  will  further  iso¬ 
late  the  controlling  factors  and  phenomena  that  would  provide  quantitative  links  to  meteorological  and 
oceanographic  conditions  in  the  scene. 
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